What makes a word : Learning base units in Japanese for speechrecognitionLaura

We describe an automatic process for learning word units in Japanese. Since the Japanese orthography has no spaces delimiting words, the rst step in building a Japanese speech recognition system is to deene the units that will be recognized. Our method applies a compound-nding algorithm, previously used to nd word sequences in English, to learning syllable sequences in Japanese. We report that we were able not only to extract meaningful units, eliminating the need for possibly inconsistent manual segmentation, but also to decrease perplexity using this automatic procedure, which relies on a statistical, not syntactic, measure of relevance. Our algorithm also uncovers the kinds of environments that help the recognizer predict phonological alternations, which are often hidden by morphologically-motivated tok-enization.